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Laboratory -on  the  task  of  restoring  noisy,  blurry  images  which  arise  in 
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algorithms  were  speeded  up  using  an  implicit  method  which  allowed  the 
constraints  to  be  enforced  for  large  time  marching  steps.  The  results, 
successfully  applied  to  real  images,  are  displayed. 
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Cognitech,  Inc.  has  used  its  state-of-the-art  algorithms  to  restore  noisy  blurry  images 
supplied  to  us  by  Dr.  Brent  Ellerbroek  of  Philips  Laboratory. 

The  goal  is  to  reconstruct  real  images  which  arise  in  aero-optic  metrology. 

Cognitech’s  technique  is  based  on  the  use  of  nonlinear  partial  differential  equations 
and  multiscale  analysis.  Specifically  we  are  given  a  noisy  blurry  image  in  the  form 

(1)  uo(x,  y)  =  iAu){x,  y)  -1-  n(x,  y)  (additive  noise) 

(2)  «o(a:,y)  =  [(Au)(a;,y)][n(x,y)]  (multiplicative  noise) 

Other  models  of  multiplicative  noise  and  speckel  noise  have  also  been  tried. 

Here  .A  is  a  linear  integral  operator  whose  point  spread  function  is  given  experimen¬ 
tally,  and  statistics  of  the  noise-mean  and  variance-  are  estimated. 

Our  original  method  was  just  to  minimize  the  total  variation  (TV)  of  the  image  subject 
to  the  constraints  induced  by  the  models  (1),  (2)  (or  otherwise).  See  [1,2,3]  for  further 
discussion. 

Recently  Dr.  Rudin,  developed  an  important  modification  of  this  algorithm  which 
appears  to  be  even  less  invasive  than  the  TV  based  restoration  while  still  finding  features 
in  blurry,  noisy  images.  Namely,  one  minimizes  the  quantity: 

(3)  = 


Here  jVuj  =  js  -  j  (^)  for  <5  >  0  where  j  is  a  positive  smoothing 

kernel  of  mass  one  and  *  denotes  convolution.  Various  values  of  6  are  chosen,  depending 
on  the  desired  level  of  sharpness.  Thus  this  method  is  denoted  multiscale  total  variation 
regularization. 

We  also  intend  to  use  some  of  the  newer  restoration  algorithms  which  we  developed 
in  [4]  which  involve  free  local  constraints  in  future  work  in  this  area. 

An  additional  new  idea  came  in  speeding  up  the  restoration  procedure  by  making  the 
algorithm  fully  implicit,  including  an  implicit  treatment  of  the  constraints.  We  wish  to 
update  an  algorithm  of  the  form: 


(4) 


<,+iA>?,  +  a^..jAlu?,l 

-k  A  (constraint)”. 


Here  A  is  a  Lagrange  multiplier,  CFL  is  the  time  step/space  step  ratio,  the  term  (constraint) 
comes  from  the  Euler-Lagrange  equations  for  the  statistical  and  blurring  constraints.  Also 
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the  afj  are  nonlinear  functions  of  (u”),  but  they  are  always  nonnegative.  Our  first  obser¬ 
vation  is  that  we  can  increase  the  CFL  to  an  arbitrary  level,  for  A  =  0,  by  simply  replacing 
the  terms  by  The  result  is  a  linear  system  which  is  uniformly  diagonally 

dominant  and  block  tridiagonal,  thus  it  is  easily  inverted  by  e.g.  approximate  factorization. 
However  A  has  to  be  chosen  so  that  a  certain  nonlinear  constraint  is  satisfied. 

We  do  this  as  follows:  We  wish  to  solve 

(5)  [7  -|-  =  tt”  -b  A  (constraint)" 

where  satisfies  a  certain  nonlinear  constraint.  We  compute 

(6a)  =  (J-b  L)-^u” 

(6b)  =  (1  +  (constraint)". 

Then  -b  Ati)”"*"^. 

We  finally  choose  A  so  that  the  nonlinear  equation  enforcing  the  constraint  is  satisfied. 

A  key  step  in  our  restoration  procedure,  suggested  by  L.  Rudin,  is  to  use  it  iteratively 
with  our  multiscale  segmentation  algorithm  using  the  method  originating  in  the  work  of 
our  consultants.  Professor  J.-M.  Morel  and  improved  significantly  at  Cognitech  by  Rudin 
and  Nordby,  in  collaboration  with  our  French  consultants  [5,6]. 

We  now  describe  our  results,  all  done  on  our  HP  735  computers,  using  a  few  min¬ 
utes  of  computing  time,  at  most.  Figure  (la)  shows  the  original  degraded  image  (SeaSat 
119a),  while  (lb)  shows  our  restoration  using  TV  regularization  with  an  additive  noise 
model.  Figure  (Ic)  shows  the  restoration  using  MTV,  assuming  a  multiplicative  noise 
model  iterated  together  with  our  multiscale  segmentation  through  which  the  constraints 
were  enforced  locally. 

Figure  (2a)  shows  the  original  degraded  image  (SeaSat  123a).  In  figure  (2b)  we  use 
our  TV  restoration  assuming  additive  noise,  while  figure  (2c)  displays  the  result  using 
MTV  restoration,  again  assuming  additive  noise. 

Figure  (3a)  shows  the  original  degraded  image  (SeaSat  081a)  while  figure  (3b)  shows 
the  restored  image  using  MTV,  iterated  segmentation  -  restoration,  assuming  a  multiplica¬ 
tive  noise  model. 

Figures  (4a,b)  show  the  analogous  sequence  for  image  SeaSat  082a. 

Figures  (5ab)  show  the  the  analogous  sequence  for  SeaSat  087a,  while  figure  (5c)  shows 
the  result  of  combining  this  with  TV  restoration. 

We  conclude  that  our  approach  is  a  very  promising  line  of  attack  for  the  restoration 
of  images  of  this  type.  Other  restoration  models  will  be  used  as  well  as  constraints  based 
on  a  speckel  noise  assumption. 
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a)  Original  Image  (Seasat  1 1 9a) 


b)  Restoration  of  Original  Image 
using  Total  Variation,  assuming 
additive  noise 


c)  Restoration  of  Original  Image 
using  Multiscale  Total  Variation 
and  Segmentation,  assuming 
multiplicative  noise 


Figure  1 


a)  Original  Image  (Seasat  123a) 


b)  Restoration  of  Original  Image 
using  Total  Variation,  assuming 
additive  noise 


c)  Restoration  of  Original  Image 
using  Multiscale  Total  Variation, 
assuming  additive  noise 


Figure  2 


a)  Original  Image  (Seasat  081a) 


b)  Restoration  of  Original  Image 
using  Multiscale  Total  Variation 
and  Segmentation,  assuming 
multiplicative  noise 


Figure  3 


a)  Original  Image  (Seasat  082a) 


b)  Restoration  of  Original  Image 
using  Multiscale  Total  Variation 
and  Segmentation,  assuming 
multiplicative  noise 


Figure  4 


a)  Original  Image  (Seasat  087a) 


b)  Restoration  of  Original  Image 
using  Multiscale  Total  Variation 
and  Segmentation,  assuming 
multiplicative  noise 


c)  Restoration  of  Original  Image 
using  Total  Variation,  Multiscale 
Total  Variation,  and  Segmentation 
assuming  multiplicative  noise 


Figure  5 


